Many compelling video processing effects can be achieved if per pixel depth information and three-dimensional (3D) camera calibrations are known. Scene-space video processing, where pixels are processed according to their 3D positions, has many advantages over traditional image-space processing. For example, handling camera motion, occlusions, and temporal continuity entirely in two-dimensional (2D) image-space can in general be very challenging, while dealing with these issues in scene-space is simple. As scene-space information becomes more and more widely available due to advances in tools and mass market hardware devices, techniques that leverage depth information will play an important role in future video processing approaches. However, the success of such methods is highly dependent on the accuracy of the scene-space information.